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-Abstract- 

We study n-player turn-based games played on a finite directed graph. For each play, the 
players have to pay a cost that they want to minimize. Instead of the well-known notion 
of Nash equilibrium (NE), we focus on the notion of subgame perfect equilibrium (SPE), a 
refinement of NE well-suited in the framework of games played on graphs. We also study 
natural variants of SPE, named weak (resp. very weak) SPE, where players who deviate 
cannot use the full class of strategies but only a subclass with a finite number of (resp. a 
unique) deviation step(s). 

Our results are threefold. Firstly, we characterize in the form of a Folk theorem the set 
of all plays that are the outcome of a weak SPE. We also establish a weaker version of this 
theorem for SPEs. Secondly, for the class of quantitative reachability games, we prove the 
existence of a finite-memory SPE and provide an algorithm for computing it (only existence 
was known with no information regarding the memory). Moreover, we show that the existence 
of a constrained SPE, i.e. an SPE such that each player pays a cost less than a given constant, 
can be decided. The proofs rely on our Folk theorem for weak SPEs (which coincide with 
SPEs in the case of quantitative reachability games) and on the decidability of MSO logic on 
infinite words. Finally with similar techniques, we provide a second general class of games for 
which the existence of a (constrained) weak SPE is decidable. 

1998 ACM Subject Classification B.6.3 [design aids]: automatic synthesis; F.1.2 [Modes of 
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Keywords and phrases multi-player games on graphs, quantitative objectives, Nash equilib¬ 
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[T] Introduction 

Two-player zero-sum infinite duration games played on graphs are a mathematical model 
used to formalize several important problems in computer science. Reactive system synthesis 
is one such important problem. In this context, see e.g. m, the vertices and the edges of the 
graph represent the states and the transitions of the system; one player models the system 
to synthesize, and the other player models the (uncontrollable) environment of the system. 
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Weak Subgame Perfect Equilibria in Quantitative Games 


In the classical setting, the objectives of the two players are opposite, i.e. the environment is 
adversarial. Modeling the environment as fully adversarial is usually a bold abstraction of 
reality and there are recent works that consider the more general setting of non zero-sum 
games which allow to take into account the different objectives of each player. In this latter 
setting the environment has its own objective which is most often not the negation of the 
objective of the system. The concept of Nash equilibrium (NE) [TT] is central to the study of 
non zero-sum games and can be applied to the general setting of n player games. A strategy 
profile is a NE if no player has an incentive to deviate unilaterally from his strategy, since he 
cannot strictly improve on the outcome of the strategy profile by changing his strategy only. 

However in the context of sequential games (such as games played on graphs), it is 
well-known that NEs present a serious weakness: a NE allows for non-credible threats that 
rational players should not carry out m- As a consequence, for sequential games, the notion 
of NE has been strengthened into the notion of subgame perfect equilibrium (SPE): a strategy 
profile is an SPE if it is a NE in all the subgames of the original game. While the notion 
of SPE is rather well understood for finite state game graphs with w-regular objectives or 
for games in finite extensive form (finite game trees), less is known for game graphs with 
quantitative objectives in which players encounter costs that they want to minimize, like 
in classical quantitative objectives such as mean-payoff, discounted sum, or quantitative 
reachability. 

Several natural and important questions arise for such games: Can we decide the existence 
of an SPE, and more generally the constrained existence of an SPE (i.e. an SPE in which 
each player encounters a cost less than some fixed value)? Can we compute such SPEs 
that use finite-memory strategies only? Whereas several work has studied what are the 
hypotheses to impose on games in a way to guarantee the existence of an SPE, the previous 
algorithmic questions are still widely open. In this article, we provide progresses in the 
understanding of the notion of SPE. We first establish Folk theorems that characterize the 
possible outcomes of SPEs in quantitative games for SPEs and their variants. We then 
derive from this characterization interesting algorithms and information on the strategies 
for two important classes of quantitative games. Our contributions are detailed in the next 
paragraph. 

Contributions First, we formalize a notion of deviation step from a strategy profile 
that allows us to define two natural variants of NEs. While a NE must be resistant to the 
unilateral deviation of one player for any number of deviation steps, a weak (resp. very weak) 
NE must be resistant to the unilateral deviation of one player for any finite number of (resp. 
a unique) deviation step(s). Then we use those variants to define the corresponding notions 
of weak and very weak SPE. The latter notion is very close to the one-step deviation property 
m- Any very weak SPE is also a weak SPE, and there are games for which there exists a 
weak SPE but no SPE. Also, for games with upper-semicontinuous cost functions and for 
games played on finite game trees, the three notions are equivalent. 

Second, we characterize in the form of a Folk theorem all the possible outcomes of weak 
SPEs. The characterization is obtained starting from all possible plays of the game and 
the application of a nonincreasing operator that removes plays that cannot be outcome of 
a weak SPE. We show that the limit of the nonincreasing chain of sets always exists and 
contains exactly all the possible outcomes of weak SPEs. Furthermore, we show how for each 
such outcome, we can associate a strategy profile that generates it and which is a weak SPE. 
Using a variant of the techniques developed for weak SPEs, we also obtain a weaker version 
of this Folk theorem for SPEs. 

Additionally, to illustrate the potential of our Folk theorem, we show how it can be 
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refined and used to answer open questions about two classes of quantitative games. The first 
class of games that we consider are quantitative reachability games, such that each player 
aims at reaching his own set of target states as soon as possible. As the cost functions 
in those games are continuous, our Folk theorem characterizes precisely the outcomes of 
SPEs and not only weak SPEs. In DE], it has been shown that quantitative reachability 
games always have SPEs. The proof provided for this theorem is non constructive since it 
relies on topological arguments. Here, we strengthen this existential result by proving that 
there always exists, not only an SPE but, a finite-memory SPE. Furthermore, we provide an 
algorithm to construct such a finite memory SPE. This algorithm is based on a constructive 
version of our Folk Theorem for the class of quantitative reachability games: we show that 
the nonincreasing chain of sets of potential outcomes stabilizes after a finite number of steps 
and that each intermediate set is an w-regular set that can be effectively described using 
MSO sentences. The second class of games that we consider is the class of games with cost 
functions that are prefix-independent, whose range of values is finite, and for which each 
value has an w-regular pre-image. For this general class of games, with similar techniques as 
for quantitative reachability games, we show how to construct an effective representation of 
all possible outcomes compatible with a weak SPE, and consequently that the existence of a 
weak SPE is decidable. In those two applications, we show that our construction also allow 
us to answer the question of existence of a constrained (weak) SPE, i.e. a (weak) SPE in 
which players pays a cost which is bounded by a given value. 

Related work The concept of SPE has been first introduced and studied by the game 
theory community. The notion of SPE has been first introduced by Kuhn in finite extensive 
form games [5]. For such games, backward induction can be used to prove that there always 
exist an SPE. By inspecting the backward induction proof, it is not difficult to realize that 
the notion of very weak SPE and SPE are equivalent in this context. 

SPEs for infinite trees defined as the unfolding of finite graphs with qualitative, i.e. 
win-lose, w-regular objectives, have been studied by Ummels in [T5]: it is proved that such 
games always have an SPE, and that the existence of a constrained SPE is decidable. 

In [5], Klimos et al. provide an effective representation of the outcomes of NEs in 
concurrent priced games by constructing a Biichi automaton accepting precisely the language 
of outcomes of all NEs satisfying a bound vector. The existence of NEs in quantitative games 
played on graphs is studied in [5]; it is shown that for a large class of games, there always 
exists a finite-memory NE. This result is extended in [3] for two-player games and secure 
equilibria (a refinement of NEs); additionally the constrained existence problem for secure 
equilibria is also shown decidable for a large range of cost functions. None of these references 
consider SPEs. 

In [5], the authors prove that for quantitative games with cost functions that are upper- 
semicontinus and with finite range, there always exits an SPE. This result also relies on a 
nonincreasing chain of sets of possible outcomes of SPEs. The main differences with our 
work is that we obtain a Folk theorem that characterize all possible outcomes of weak SPEs 
with no restriction on the cost functions. Moreover we have shown that our Folk theorem 
can be made effective for two classes of quantitative games of interest. Effectiveness issues 
are not considered in [5]. Prior to this work, Mertens shows in m that if the cost functions 
are bounded and Borel measurable then there always exists an e-NE. In [6], Fudenberg et al. 
show that if the cost functions are all continous, then there always exists an SPE. Those 
works were recently extended in |14] by Le Roux and Pauly. 

Organization of the article In Section we present the notions of quantitative game, 
classical NE and SPE, and their variants. In Section we propose and prove our Folk 
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Theorems for weak SPEs and for SPEs. In Section we provide an algorithm for computing 
a finite-memory SPE for quantitative reachability games, and a second algorithm to decide 
the constrained existence of an SPE for this class of games. In Section we show that the 
existence of a (constrained) weak SPE is decidable for another class of games. A conclusion 
and future work are given in the last section. 


In this section, we recall the notions of quantitative game, Nash equilibrium, and subgame 
perfect equilibrium. We also introduce variants of Nash and subgame perfect equilibria, and 
compare them with the classical notions. 

2.1 Quantitative Games 

We consider multi-player turn-based non zero-sum quantitative games in which, for each 
infinite play, players pay a cost that they want to minimizej^ 

► Definition 1. A quantitative game is a tuple Q = (H, F, (F)ign, A, A) where: 

H n is a finite set of players, 

H F is a finite set of vertices, 

" is a partition of F such that F is the set of vertices controlled by player i G 11, 

H A C F X F is a set of edges, such thalj^for all v G V, there exists v' G V with (n, v') G E, 
™ A = (Ai)ign is a cost function such that Xi : F“ —> K U {-boo} is player i cost function. 

A play of Q is an infinite sequence p = popi... G F'^ such that {pi,pi+i) G E for all i G N. 
Histories of Q are finite sequences h = ho ... hn G F"*" defined in the same way. The length 
\h\ of h is the number n of its edges. We denote by First(h.) (resp. Last(/i)) the first vertex 
ho (resp. last vertex hn) of h. Usually histories are non-empty, but in specific situations it 
will be useful to consider the empty history e. The set of all histories (ended by a vertex 
in F) is denoted by Hist (by Histi). A prefix (resp. suffix) of a play p is a finite sequence 
po ... Pn (resp. infinite sequence pnPn+i ■ ■ ■) denoted by p<„ or p<„+i (resp. p>„). We use 
notation h < p when a history h is prefix of a play p. Given two distinct plays p and p', their 
longest common prefix is denoted by p^p'. 

When an initial vertex wq € is fixed, we call (Q,vo) an initialized quantitative game. A 
play (resp. a history) of {Q, wq) is a play (resp. history) of Q starting in no. The set of histories 
h G Hist (resp. h G Histi) with First (h) = no is denoted by Hist (no) (resp. Histi(no)). In the 
figures of this article, we will often unravel the graph of the game (Q,vo) from the initial 
vertex vq, which ends up in an infinite tree. 

Given a play p G V‘^, its cost is given by A(p) = (Ai(p))ign- In this article, we are 
particularly interested in quantitative reachability games in which Xi{p) is equal to the 
number of edges to reach a given set of vertices. 

► Definition 2. A quantitative reachability game is a quantitative game Q such that the cost 
function A : F'^ —>■ (N U {-boo})^ is defined as follows. Each player i has a target set C F, 
and for each play p = poPi ... of the cost Xfip) is the least index n such that p„ G F if it 
exists, and -boo otherwise. 


2 Preliminaries and Variants of Equilibria 


^ Alternatively, players could receive a payoff that they want to maximize. 
^ Each vertex has at least one outgoing edge. 
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Notice that the cost function A of a quantitative game is often defined from |n|-uples of 
weights labeling the edges of the game. For instance, in inf games, Xi{p) is equal to the 
inhmum of player i weights seen along p. Some other classical examples are liminf, limsup, 
mean-payoff, and discounted sum games ^ . In case of quantitative reachability on graphs 
with weighted edges, the cost \i{p) for player i is replaced by the sum of the weights seen 
along p until his target set is reached. We do not consider this extension here. Notice that 
when weights are positive integers, replacing each edge with cost c by a path of length c 
composed of c new edges allows to recover Definition 

Let us recall the notions of prefix-independent, continuous, and lower- (resp. upper-) 
semicontinuous cost functions. Since V is endowed with the discrete topology, and thus V‘^ 
with the product topology, a sequence of plays (pn)nGN converges to a play p = lim„_,.oo Pn 
if every prefix of p is prefix of all except, possibly, of finitely many of them. 

► Definition 3. Let be a player i cost function. Then 
H \i is prefix-independent if Xfihp) = Xfip) for any history h and play p. 

H Xi is continuous if whenever lim„_>oo Pn = P, then lim„_>oo Xfipn) = Xfip). 

H Xi upper-semicontinuous (resp. lower-semicontinuous) if whenever lim„_>oo Pn = P, then 

limsup„^^ Ai(p„) < Aj(p) (resp. liminf„^oo Ai(p„) > Xfip)). 

For instance, the cost functions used in liminf and mean-payoff games are prefix-independent, 
contrarily to the case of inf games. Clearly, if Xi is continuous, then it is upper- and lower- 
semicontinuous. For instance, the cost functions of liminf and mean-payoff games are neither 
upper-semicontinuous nor lower-semicontinuous, whereas cost functions of discounted sum 
games are continuous. The cost functions A^, i G 11, used in quantitative reachability games 
can be transformed into continuous ones as follows [T]: A'(p) = 1 — if Xfip) < -foo, 

and A'(p) = 1 otherwise. 

2.2 Strategies and Deviations 

A strategy a for player i G 11 is a function a : Histi —>■ V assigning to each historj0/iu G Histi 
a vertex v' = a(hv) such that {v,v') G E. In an initialized game (5,uo), a is restricted to 
histories starting with vq. A player i strategy a is positional if it only depends on the last 
vertex of the history, i.e. a{hv) = cr(pj) for all hv G Histi. It is a finite-memory strategy if it 
needs only finite memory of the history (recorded by a finite strategy automaton, also called 
a Moore machine). A play p is consistent with a player i strategy a if Pfc+i = cr{p<k) for all k 
such that pk G Vi- A strategy profile of ^ is a tuple a = ((Ti)ign of strategies, where each ai is 
a player i strategy. It is called positional (resp. finite-memory) if all ai, i G U, are positional 
(resp. finite-memory). Given an initial vertex vq, such a strategy profile determines a unique 
play of (Q,vo) that is consistent with all the strategies. This play is called the outcome of a 
and is denoted by {a)yg. 

Given ai a player i strategy, we say that player i deviates from ai if he does not stick 
to ai and prefers to use another strategy ct'. Let ct be a strategy profile. When all players 
stick to their strategy ai except player i that shifts to tr', we denote by {a'i,a-i) the derived 
strategy profile, and by (ct', a-i)y„ its outcome in (Q, vq). In the next definition, we introduce 
the notion of deviation step of a strategy a) from a given strategy profile a. 
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In this article we often write a history in the form hv with v G V to emphasize that v is the last 
vertex of this history. 
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► Definition 4. Let (Q,vo) be an initialized game, ct be a strategy profile, and a' be a player 
i strategy. We say that cr' has a hv-deviation step from a for some history hv € Histi(no) 
with V G Vi, if 

hv < {a'i,a-i)vQ and ai{hv) ^ a'i{hv). 

Notice that the previous definition requires that hv is a prefix of the outcome (cr', it 

says nothing about a[ outside of this outcome. A strategy tr' can have a finite or an infinite 
number of deviation steps in the sense of Definition A strategy with three deviation 
steps is depicted in Figure (left) such that each /ifeUfc-deviation step from tr, 1 < A: < 3, is 
highlighted with a dashed edge. We will come back to this figure later on. 



— h2V2-{<T\h^v2)v!2 


M Figure 1 A strategy ct' with a finite number of deviation steps. 

In light of Definition 1^ we introduce the following classes of strategies. 

► Definition 5. Let {G,vo) be an initialized game, and ct be a strategy profile. 

B A strategy cr) is finitely deviating from a if it has a finite number of deviation steps from 
a. 

H It is one-shot deviating from a if it has a rig-deviation step from a, and no other deviation 
step. 

In other words, a strategy cr' is finitely deviating from tf if there exists a history hv < 
{a^,a-i)vg such that for all h'v', hv < h'v' < {a[,a-i)yg, we have a[{h'v') = ai{h'v') (cr' acts 
as tTi from hv along {a[,a-i)yg). The strategy cr' is one-shot deviating from a if it differs 
from Ui at the initial vertex ug, and after ug acts as Ui along As for Definition|^ 

the previous definition says nothing about ct' outside of (cr',cr_i)„p. Clearly any one-shot 
deviating strategy is finitely deviating. The strategy of Figure [T] is finitely deviating but not 
one-shot deviating. 
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2.3 Nash and Subgame Perfect Equilibria, and Variants 

In this paper, we focus on subgame perfect equilibria and their variants. Let us first recall 
the classical notion of Nash equilibrium. A strategy profile a in an initialized game is a Nash 
equilibrium if no player has an incentive to deviate unilaterally from his strategy, since he 
cannot strictly decrease his cost when using any other strategy. 

► Definition 6. Given an initialized game a strategy profile a — (cri)ign of {G,vo) 

is a Nash equilibrium (NE) if for all players z G Lf, for all player i strategies cr', we have 

We say that a player i strategy cr' is a profitable deviation for i w.r.t. a if Ai((af a-i}yg) < 
Ai((d'}yg). Therefore ct is a NE if no player has a profitable deviation w.r.t. a. 

Let us propose the next variants of NE. 

► Definition 7. Let (tJ,uo) be an initialized game. A strategy profile (f is a weak NE (resp. 

very weak NE) in (G,vo) if, for each player z S Lf, for each finitely deviating (resp. one-shot 
deviating) strategy tr' of player i, we have a-f)yg) > \i{{u)yg). 

► Example 8. Consider the two-player quantitative game depicted in Figure]^ Circle (resp. 
square) vertices are player 1 (resp. player 2) vertices. The edges are labeled by couples of 
weights such that weights (0,0) are not specified. For each player z, the cost Xi{p) of a play 
p is the weight of its ending loop. In this simple game, each player i have two positional 
strategies that are respectively denoted by Oi and cr' (see Figure]^. 


i 



H Figure 2 A simple two-player quantitative game 


The strategy profile (cri,cr 2 ) is not a NE since a\ is a profitable deviation for player 1 
w.r.t. (a\^a'f) (player 1 pays cost 1 instead of cost 3). This strategy profile is neither a weak 
NE nor a very weak NE because in this simple game, player 1 can only deviate from cri by 
using the one-shot deviating strategy cr). On the contrary, the strategy profile (cri,cr 2 ) is a 
NE with outcome of cost (3,3). It is also a weak NE and a very weak NE. 

By definition, any NE is a weak NE, and any weak NE is a very weak NE. The contrary 
is false: in the previous example, (cr), 0 - 2 ) is a very weak NE, but not a weak NE. We will see 
later an example of game with a weak NE that is not an NE (see Example [I^ . 

The notion of subgame perfect equilibrium is a refinment of NE. In order to define it, we 
need to introduce the following notions. Given a quantitative game Q — (11, V, (Vi)ien, E, A) 
and a history h of G, we denote by G\h the game G\h = (n, (t4)iGn, 75, Af,_^) where 

= Khp) 
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for any play of and we say that G\h is a subgame of G- Given an initialized game (G,vo), 
and a history hv € Hist(tio), the initialized game (Gfh,y) is called the subgame of (0,uo) 
with history hv. Notice that {G,vo) can be seen as a subgame of itself with history hvo such 
that h = e. Given a player i strategy a in (G,vo), we define the strategy a^h in iG\h,v) as 
aih{h') = (j{hh') for all histories h' G Histi(u). Given a strategy profile a = (cri)ign, we use 
notation a^h for and {d’^h)v is its outcome in the subgame {G\h,v)- 

We can now recall the classical notion of subgame perfect equilibrium; it is a strategy 
profile in an initialized game that induces a NE in each of its subgames. In particular, a 
subgame perfect equilibrium is a NE. 

► Definition 9. Given an initialized game (G,vo), a strategy profile a of {G,vo) is a subgame 
perfect equilibrium (SPE) if a^h is a NE in {G\h,v), for every history hv G Hist(uo). 

As for NE, we propose the next variants of SPE. 

► Definition 10. Let (t/,wo) be an initialized game. A strategy profile it is a weak SPE 
(resp. very weak SPE) if aih is a weak NE (resp. very weak NE) in {G\h,v), for all histories 
hv G Hist(wo)- 

► Example 11. We come back to the game depicted in Figure We have seen before 

that the strategy profile (cri,tT 2 ) is a NE. However it is not an SPE. Indeed consider the 
subgame (0|'„o,r'2) of (5,no) with history VqV 2 . In this subgame, cr^ is a profitable deviation 
for player 2. One can easily verify that the strategy profile is an SPE, as well as a 

weak SPE and a very weak SPE, due to the simple form of the game. 

The previous example is too simple to show the differences between classical SPEs and 
their variants. The next example presents a game with a (very) weak SPE but no SPE. 

► Example 12. Consider the initialized two-player game {G,vo) in Figure]^ The edges are 
labeled by couples of weights, and for each player i the cost Xi{p) of a play p is the unique 
weight seen in its ending cycle. With this definition, Xi{p) can also be seen as either the 
mean-payoff, or the liminf, or the limsup, of the weights of p. It is known that this game has 
no SPE [TH] . 

Let us show that the positional strategy profile tr depicted with thick edges is a very 
weak SPE. Due to the simple form of the game, only two cases are to be treated. Consider 
the subgame with h G (vq^^i)*) and the one-shot deviating strategy (t( of player 1 

such that cr((uo) = W 2 - Then {d'^h)vo = VoViv'f and {o'[,a 2 \h)vo = 'Co'C 2 i showing that a[ is 
not a profitable deviation for player 1. One also checks that in the subgame {G\hT'Ui) with 
h G (uoni)*no, the one-shot deviating strategy cr^ of player 2 such that (J 2 {vi) = vq is not 
profitable for him. 

Similarly, one can prove that ct is a weak SPE (see also Proposition [I^ hereafter). Notice 
that a is not an SPE. Indeed the strategy CT 2 such that a 2 {hvi) = vq for all h, is a profitable 
deviation for player 2 in {G,vo). This strategy is (of course) not finitely deviating. Finally 
notice that ct is a weak NE that is not an NE. 

From Definition any SPE is a weak SPE, and any weak SPE is a very weak SPE. The 
next proposition states that weak SPE and very weak SPE are equivalent notions, but this is 
no longer true for SPE and weak SPE as shown previously by Example |12| 
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In this article, we will always use notation X(hp) instead of Afh(/9). 
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(1,2) (0,1) 



( 2 . 0 ) 


M Figure 3 A two-player game with a (very) weak SPE and no SPE. For each player, the cost of a 
play is his unique weight seen in the ending cycle. 

► Proposition 13. h Let (Q,vo) be an initialized game, and a be a strategy profile. Then a 
is a weak SPE iff a is a very weak SPE. 

H There exists an initialized game (Q,vo) with a weak SPE but no SPE. 

Before proving this proposition, we would like to come back to the definition of deviation 
step (Definition]^, and explain it now with the concept of subgame. Given an initialized game 
{Q,vo), a strategy profile a, and a player i strategy cr', we recall that cr' has a /in-deviation 
step from a for some hv G Histi(wo)) if hv < {al,a-i)y„ and <Ti{hv) a[{hv). Equivalently, 
(jl has a /in-deviation step from a iff 

h-{a\h)v^{<7'i,<7-i)y„ = hv. 

This alternative vision of deviation step is depicted in Figure]^ (right) for a strategy with 
three deviation steps. For instance, for history h 2 V 2 , we have h 2 V 2 < {(j[,a-i)yg and 
o'i{h 2 V 2 ) 7 ^ 0 ''(/i 2 ^i 2 )j or equivalently h 2 -{d'^h 2 )v 2 ^{’^i:^-i)vo = Notice that there is no 
intermediate deviation step between the hiUi-deviation step and the / 12 ' 112 -deviation step 
since h 2 - {a\h 2 )v 2 = hivi - {a\hxvf)v[ as indicated in the figure. Similarly the /iirii-deviation 
step is the first one because hi - {d\hf}vi = {^)vo, and the /laUa-deviation step is the third 
one because h^ - = h 2 V 2 - {h\h 2 V 2 )v'.^- The latter deviation step is the last one because 

^—i)vO ~ h^V^- 

Proof of Proposition |13[ This proof is based on arguments from the one-step deviation 
property used to prove Kuhn’s theorem [9]. Let it be a very weak SPE, and let us prove that 
it is a weak SPE. As a contradiction, assume that there exists a subgame v) such that 

the strategy profile is not a weak NE. This means that there exists a player i strategy cr' 

in (t/f/j,n) such that cr' is finitely deviating from and 

Xi{hp) > \i{hp'), (1) 

where p = {d\h)y and p' = (cr', cr_q^)„. Let us consider such a strategy cr' with a minimum 
number n of deviation steps from (itand let gkVk, 1 < fc < n, be the histories in Histj(f) 
such that cr' has a gfcUfc-deviation step from {u\h)v Let us consider the subgame {Q\hgy,Vn) 
(see Figure |4|. In this subgame, is not a profitable one-shot deviating strategy as it is a 
very weak SPE. In other words, for g = (it|,ft,g„)^„ and g’ = {o'i^g , we have 

Xiihg^g) < \i{hgng'). ( 2 ) 

Notice that n > 2. Indeed, if n = 1, then p = gig, p' = gig', and Xi{hp) < Xi{hp') by 
(21. Therefore ct' is not a profitable deviation in {G\h,v), in contradiction with its definition 
(11. We can thus construct a strategy r' from cr' such that these two strategies are the same 
except in the subgame {G\hg„i'Vn) where and Ui^kgy coincide. In other words r' has 
n — 1 deviation steps from {a\h)v, that are exactly the deviation steps, 1 < A: < n — 1, 
of cr'. Moreover, in the subgame {G\h,v), we have {tI, a-i^ff)y = gng, and 

Xi{hgng) < XiQigng) < Xi{hp) 
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M Figure 4 A strategy ct' with a minimum number n of deviation steps. 
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by 0, 0, and gnQ' = p' ■ It follows that r' is a finitely deviating strategy that is profitable 
for player i in {G\h,v), with less deviation steps than cr', a contradiction. This completes 
the proof of the first statement of Proposition 13 For the second statement, it is enough to 


consider Example 12 


Under the next hypotheses on the game or the costs, the equivalence between SPE, weak 
SPE, and very weak SPE holds. The first case, when the cost functions are continuous, is a 
classical result in game theory, see for instance [7]; the second case appears as a part of the 
proof of Kuhn’s theorem . 

► Proposition 14. Let (G,vo) be an initialized game, and a be a strategy profile. 

H If all cost functions Xi are continuous, or even upper-semicontinuouJ^ then a is an SPE 
iff a is a weak SPE iff a is a very weak SPE. 

™ If G is a finite tre^ then a is an SPE iff a is a weak SPE iff a is a very weak SPE. 

Proof. We only prove the first statement for cost functions Xi that are upper-semicontinuous. 
Let ft be a strategy profile in an initialized game {G,Vq). By Proposition |13[ it remains to 
prove that if it is a weak SPE, then it is an SPE, i.e., for each subgame {G\h,v), the strategy 
profile a^h is a NE. Let ct' be a player i strategy in {G\hjv). If cr' is finitely deviating, then it 
is not a profitable deviation for player i w.r.t. a^h by hypothesis. Therefore, suppose that 

P — i’^i^ ^—i\h)'» ~ 9 i 92 ■ ■ ■ gn ■ ■ ■ 

such that cr' has a /i„-deviation step from a\h for all n > 1, where ho = e and hn = 

For each n, we define a finitely deviating strategy rfi such that its deviation steps are 
the first n deviation steps of ct', that is, rfi and cr' are equal except in the subgame 
(0r/t„,First(g„+i)) where By hypothesis, rfi is not a profitable deviation, 

and thus for = {rfi we have 


Aj(h*(cr['/,)^) ^ Xfihpffj. (3) 

As lim„_>oo hpn = hp' and Xi is upper-semicontinuous, we get limsup„_,,oo Xfihpn) < Xfihp'). 
Therefore by 0, Xfih- {a\h)v) < Xfihp') showing that cr' is not a profitable deviation for 
player i w.r.t. a\h. ◄ 


Recall that discounted sum games and quantitative reachability games are continuous. 
Thus for these games, the three notions of SPE, weak SPE and very weak SPE, are equivalent. 

► Corollary 15. Let (G,vo) be an initialized quantitative reachability game, and a be a 
strategy profile. Then a is an SPE iff a is a weak SPE iff a is a very weak SPE. 

On the opposite, the initialized game of Figurej^has a weak SPE but no SPE. Its cost func¬ 
tion A 2 is not upper-semicontinuous as lim„_,.oo(wo'yi )"^^3 = and lim„_,.oo X 2 {{voViffv'^) 

= 1 > 0 = A2((noz;i)“). 


® In games where the players receive a payoff that they want to maximize, the hypothesis of upper- 
semicontinuity has to be replaced by lower-semicontinuity. 

® In a finite tree game, the plays are finite sequences of vertices ending in a leaf and their cost is 
associated with the ending leaf. An example of such a game is depicted in Figure]^ 
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Folk Theorems 

3.1 Folk Theorem for Weak SPEs 

In this section, we characterize in the form of a Folk Theorem the set of all outcomes of 
weak SPEs. Our approach is inspirecQ by work where a Folk Theorem is given for the set 
of outcomes of SPEs in games with cost functions that are upper-semicontinuous and have 
finite range. In this aim we define a nonincreasing sequence of sets of plays that initially 
contain all the plays, and then loose, step by step, some plays that for sure are not outcomes 
of a weak SPE, until finally reaching a fixpoint. 

Let (^jUo) be a game. For an ordinal a and a history hv G Hist(z;o), let us consider the 
set Pa(hv) = {p I p is a potential outcome of a weak NE in {G\h,v) at step a}. This set is 
defined by induction on a as follows: 

► Definition 16. Let {G,vo) be a quantitative game. The set Va(hv) is defined as follows 
for each ordinal a and history hv € Hist(uo): 

B For a = 0, 

Pa(hv) = {p I p is a play in (0fft,u)}. (4) 

H For a successor ordinal a + 1, 

Pa + l(hv) = Pa(hv) \Ea(hv) (5) 

such that p G Ea(hv) (see Figureiff 

_ there exists a history h', hv <h' < hp, and Last(/i') G Vi for some i, 

_ there exists a vertex v', h'v' ft hp, 

- such that Vp' G Pa{h'v'): \i{hp) > \i{h'p'). 

B For a limit ordinal a: 

Pc,{hv) = f| Pp{hv). (6) 

/3<a 

Notice that an element p of Pa{hv) is a play in {G\h,v) (and not in (I/,fo))- Therefore it 
starts with vertex v, and hp is a play in (t/,uo). For a + 1 being a successor ordinal, play 
p G Ea{hv) is erased from P„(/iu) because for all p' G Pa{h'v'), player i pays a lower cost 
\i(h'p') < Xi(hp), which means that p is no longer a potentiel outcome of a weak NE in 
iG\h,v). 

The sequence (PQ(hr!))Q, is nonincreasing by definition, and reaches a fixpoint in the 
following sense. 

► Proposition 17. There exists an ordinal a* such that Pa^{hv) = Pa^+i{hv) for all 
histories hv G Hist(uo). 

Proof. Let us fix a history hv G Hist(z;o)- The sequence {Pa{hv))a reaches a fixpoint as soon 
as there exists a such that Pa{hv) = Pa+i{hv). Indeed it follows that Pa+l{hv) — Pa+ 2 {hv) 
and then Pa{hv) = Pp{hv) for all /3 > a. As the sequence {Pa{hv))a is nonincreasing, 
this happens at the latest with a being equal to the cardinality of Po{hv). Therefore with 
a* = I I being an ordinal greater than or equal to the cardinality of the set of all plays of 
G, we get Pa,{hv) = Pa,+i{hv) for all hv G Hist(wo)- 

In the sequel, a* always refers to the ordinal mentioned in Proposition |I7[ 

Our Polk Theorem for weak SPEs is the next one. 
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Our approach is however different. 
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U Figure 5 p € Ea{hv). 


► Theorem 18. Let {G,vo) be a quantitative game. There exists a weak SPE in (G,vo) with 
outcome p iffPa,{hv) ^ 0 for all hv G Hist(t!o), and p G PQ,,(t!o). 

Before proving this theorem, we illustrate it with an example. 

► Example 19. Consider the example of Figure]^ Clearly, as Po{hv2) only contains the 
play ^2 , then Pa{hv2) = {^2 } for all a. Similarly Paihvs) = {ug } for all a. Let us detail 
the computation of Pq,(/iuo) and Pa{hvi). 

™ a = 0. For history huo, we have p = G Eo(huo), since Xi{hp) > Xi{hvop') 

where p' = V2 is the unique play of Po{hvoV2). Similarly, for all u > 1 , we have 
p = (uoUi)”uoU2 G Eo(/iuo), since X2{hp) > X2{hvoVip') where p' = U3 is the unique play 
of Po(huoUiU3). Thus Eo(/iuo) = {(uoUi)“} U {voVi)~^vqV2 and 

Pl{hvo) = {V0V2} U (uoUi) + U3 . 

For history hvi, with the same kind of computations, we get Eo(hui) = {(uiuo)*^} U 
{viVq)'^V2 and 

Pi{hvi) = vi{voVi)*v'^. 

H a = 1. For history hvo, we have p = V0V2 G Ei(/iz;o). Indeed Xi{hp) > Xi{hvop') for all 
p' G Pi(/iuoUi) = vi{voVi)*v^. Notice that at the previous step, p = V0V2 ^ Eo(/iuo). 
Indeed Pi(/iUoUi) C Po(/iUoUi), and Xi{hp) < Ai(huop') for p' = (uiUo)“ G Po(/iuoUi)|^ 
Play VqV2 is the only one that is removed from Pi(huo), and no play can be removed 
from Pi(hui). Therefore: 

P2(huo) = : P2{hvi) = Vi{voVi)*V3. 
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This shows that as the sequence (Pa(ha))a is nonincreasing for each hv, a play that is not removed 
from some Pa{hv) can be removed later from some P/ 3 (/ia) with fi > a. 
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™ a = 2. One checks that P 3 (/ii;o) = P 2 (^i'o), and P 3 (/i'(;i) = 'P 2 {hvi). Hence the 
fixpoint is reached with a* = 2, with Pa,{hvo) = (i;o'i'i )~''^^3 j Pa.ihvi) = Di(tio'yi)*'C 3 , 
Pa,{hv 2 ) = {V 2 }, and Pa^ihv^) = }. Therefore, the set of outcomes of weak SPEs 

in this game is equal to {voVi)~^v^. The weak SPE depicted in Figure]^ has outcome 

VoViV"^. 

The proof of Theorem follows from Lemmas and 

► Lemma 20. If {G,vo) has a weak SPE a, then Pa^{hv) ^ 0 for all hv G Hist('(;o), and 
{a')vQ G PcK^(r^o)- 

Proof. Let us show, by induction on a, that {d-^h)v & Pa{hv) for all hv G Hist(uo). 

For a = 0, we have {a\h)v G Pa{hv) by definition of Po(/ir’). 

Let a + 1 be a successor ordinal. By induction hypothesis, we have that {a\h)v G Pa{hv) 
for all hv G Hist(uo)- Suppose that there exists hv such that {aih)v ^ Pa+i(/i^^), i.e. 
{o'\h)v G Eia{hv). This means that there is a history h' = hg G Histi for some i G H 
with hv < h' < hp, and there exists a vertex v' with h'v' hp, such that Vp' G Paih'v'), 
\i{h- {d\h)v) > \{h'p')- In particular, by induction hypothesis 

\{h-{a\h)v) > h{h'■{a\h')v')- ( 7 ) 

Let us consider the player i strategy cr' in {Q\h,v) such that is consistent with u'. 

Then cr' is a finitely deviating strategy with the (unique) p-deviation step from This 
strategy is a profitable deviation for player i in ?;) by Q, a contradiction with a being 
a weak SPE. 

Let a be a limit ordinal. By induction hypothesis G Pp(/iz;),V/3 < a. Therefore 

{a^h)v &Paihv) = r\^^^Pl3{hv). ◄ 

► Lemma 21. Suppose that Pa,{hv) 0 for all hv G Hist(z;o): and let p G Pq^(z)o). Then 
{G,Vq) has a weak SPE with outeome p. 

Proof. We are going to show how to construct a very weak SPE a (and thus a weak SPE 
by Proposition |13[ ) with outcome p. The construction of a is done step by step thanks to 
a progressive labeling of the histories hv G Hist(r!o). Let us give an intuitive idea of the 
construction of a. Initially, we partially construct a such that it produces an outcome in 
{Q, Vq) equal to p G Pq,, (z^o); we also label each non-empty prefix of p by p. Then we consider 
a shortest non-labeled history h'v'^ and we correctly choose some p' G Pa,{^'v') (we will 
see later how). We continue the construction of a such that it produces the outcome p' in 
{Q\h'^ v'), and for each non-empty prefix g of p', we label h'g by p' (notice that the prefixes 
of h' have already been labeled by choice of h'). And so on. In this way, the labeling is a 
map 7 : Hist(uo) —> PaS^n) that allows to recover from h'g the outcome p' of a^h' in 
iG\h>, v') of which g is prefix. Let us now go into the details. 

Initially, none of the histories is labeled. We start with history vq and the given play 
p G Pq,,(?;o)- The strategy proHle a is partially defined such that (fT)„g = p, that is, if 
p = popi..., then <Ti(p<n) = Pn+i for all p„ G F) and z G H. The non-empty prefixes h oi p 
are all labeled with 7 (h) = p. 

At the following steps, we consider a history h'v' that is not yet labeled, but such that 
h' has already been labeled. By induction, 'y{h') = {(7ih)v such that hv < h'. Suppose that 
Last(/i') G Vi, we then choose a play p' G Pa, (h'v') such that (see Figure 

>>i(h-(a-fh}v) < Xi(h'p'). 


( 8 ) 
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M Figure 6 Construction of a very weak SPE a. 


Such a play p' exists for the following reasons. By induction, we know that G Pa^{hv). 

Since Pa,{hv) = Pa,+i{hv) by Propositionwe have {a\h)v ^ Pa^ihv), and we get the 
existence of p' by definition of Eq^(/iu). We continue to construct a such that {a^h')v' = p'■, 
i.e. if p' = p'qp\ ..then Oi(h!p'^^ = /I'Pn+i P'n ^ i G 11. For all non-empty 

prefixes g oi p', we define ^{h'g) = p' (notice that the prefixes of h' have already been 
labeled). 

Let us show that the constructed profile tf is a very weak SPE. Consider a history 
hv G Histi for some f G 11, and a one-shot deviating strategy cr) from a\h in the subgame 
iG\h,v). Let v' be such that (t'(u) = v'. By definition of a, we have ^{hv) = {^\g)u for some 
history gu < hv and h-{a\h)v = g'{^\g)u] and we have also ^{hvv') = {a'^hv)v'- Moreover 
< >^t{hv{a^hv)v') by 0, and Xi{hv-{d-^hv)v') = (cr', cr_if^)„) because tr' is 

one-shot deviating. Therefore 

Aj (/i • (fTf^^)u) Ai (^ • (fjf^^)^) ^ Xii^hv ‘ - A^ (/l • (fJ^ , (T_2 j.^).y ) 

which shows that a^h is a very weak NE in {G\h-, v). Hence d is a very weak SPE, and thus 
also a weak SPE. ◄ 

The next lemma will be useful in Sections and It states that if a play p belongs 
to Pa{hv), then each of its suffixes pi also belongs to Pa{hhiVi) such that hipi = p and 
Vi = First(pi). 

► Lemma 22. Let p G Pa{hv). Then for all hipi = p, we have pi G Pa{hhiVi) with 
vi = First(pi). 


Proof. The proof is by induction on a. The lemma trivially holds for a = 0 by definition of 

Po(M- 
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Let a + 1 be a successor ordinal. Let p G Pa+i(hv) and hipi = p with vi = First(pi). 
As Pa+i{hv) C Pa.{hv), by induction hypothesis, we have pi G Pa{hhiVi). Suppose that 
Pi G Ea(hhiVi) (hence using a history h' and a vertex v' as in Definition 161. Then one can 
easily check by definition oiE^ihUiVi) that p G Ea(hv) (by using the same h' and u'), which is 
a contradiction with p G Pa+i(hv). Therefore pi G Pa(hhiVi) \ Ea(hhiVi) = Pa+i(hhiVi). 

Let a be a limit ordinal, and suppose that p G Pa(hv). As p G Pp(hv) for all /3 < a, 
we have pi G Pp(hhiVi) by induction hypothesis. It follows that pi G Pa(hhiVi) = 
np<aP/3(^^iVi)- ^ 


3.2 Folk Theorem for SPEs 


In this section, as for weak SPEs, we characterize in the form of a Folk Theorem the set 
of all outcomes of SPEs. Nevertheless, we here need a more complex characterization with 
adapted sets P'^(hv), and this characterization only holds for cost functions that are upper- 
semicontinuousj^ The main difference appears in the definition of sets E'^(kv) that will be 
used in place of E„(/in). Indeed we will see that the set Pa(h'v') of Figure has to be 
replaced by a more complex set D^’®(h'ti'). 

Let (QjVg) be a game. For an ordinal a and a history hv G Hist (no), as in the previous 
section, we consider the set P'^{hv) = {p \ p is a potential outcome of a NEj^in 
at step a}. In order to define these sets P'^{hv), we need to introduce new definitions. In 
Definition we have introduced the notion of deviation step of a strategy from a given 
strategy profile. We here propose another concept of deviation step in relation with two 
plays (see Figure [^. 

► Definition 23. Let h'v' G Hist(r;o) and p' G {G\hG'<^')- Let hiUi,/i 2 U 2 G Hist(wo) with 
ui,U 2 G V, and pi in P'^(hiUi). We say that p' has a h 2 U 2 -deviation step from pi if 
h'v' < hiui < h 2 U 2 < h'p' and h'p'^hipi = / 12 . 


Let us denote by EL{h'v') the set of all histories / 12 M 2 such that h'v' < h 2 U 2 - Given a 
player i and H C H(/iu), the next definition introduces the notion of (H, i)-decomposition of 
a play p'. Such a play has a finite or infinite number of deviation steps such that for each 
h„u„-deviation step, the history belongs to H. Figure [^illustrates the second case of 
Definition 24, with the deviation steps highlighted with dashed edges. This definition also 
introduces the set D^’®(h'u') composed of plays with a maximal (i?, i)-decomposition. 


► Definition 24. Let h'v' G Hist(t)o) and p' G Let * be a player, and H C EL{h'v'). 

H p' has an infinite {H, i)-decomposition p' = gig 2 ... gn ■ ■ ■ if 
- for all n > 1, Last( 5 „) G Vi 

for all n > 2, G H, and p' has a /i„M„-deviation step from some Pn-i G 

Pq, {dn— 1 ^n—1) 

where hi = /i', hn+i = hnPm and Un = First(g„) Vn > 1. 

H p' has a finite {H, i)-decomposition p' = gig 2 ■ ■ ■ gmQ' if 
for all n, 1 < n < m, Last( 5 „) G Vi 

.. for all n, 2 < n < TO + 1, G H, and p' has a h„u„-deviation step from some 

Pn—l ^ — 1 ) 


® For games where players receive a payoff that they want to maximize, a similar Folk Theorem also 
exists for lower-semicontinuous cost functions, 
instead of a weak NE 
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hipi 


h' p' 


H Figure 7 Deviation step. 


where hi = h', hn+i = hng-m Un = First((/„) Vn, 1 < n < m, and u^+i = First(gi'). 

We denote by the set of plays p' with a maximal decomposition in the 

following sense: 

H p' has an infinite (H, i)-deconiposition, or 

H p' has a finite (i7, i)-decomposition p' = gig 2 ■ ■ ■ gnip' , and there exists no p" with 
a (i7, i)-decomposition p" = gig 2 ■ ■ ■ gm' ■ • ■ or p" = gig 2 ■ ■ -gm'Q" such that p'''p” = 
5152 • ■ •5m-tl- 

In the previous definition, H can be chosen finite, infinite, or empty. If = 0, then for 
all ten, for each p' € D^’*(h'n'), p' has no deviation step and thus p' G P'^{h'v'). This 
means that = P'^{h'v') in this case. 

We are ready to define the sets by induction on a. The definition is similar to 

the one of Pa{hv), except that when we erase p G P'^{hv) from P^(/in), we use some set 
P>^^\h'v') in place of P'a,{h'v'): 

► Definition 25. Let be a quantitative game. The set P'^{hv) is defined as follows 

for each ordinal a and history hv G Hist(i;o): 

H For a = 0, 

= {P I P is a play in {Gih, 'c)}- (9) 

H For a successor ordinal a -I- 1, 

p;+i(H = Pa(H\E;(H (10) 

such that p G Ei'^{hv) iff 

_ there exists a history h', hv < h' < hp, and Last(/i') G Vi for some i, 

_ there exists a vertex v\ h'v' ft hp, 

_ there exists H C Hiji'v'), 

- such that Vp' G {h'v')-. Xi{hp) > Xi{h'p'). 

(see Figurej^where (hv) is replaced by P'^(hv), and P'^(h'v') is replaced by D^’®(h'i;')). 
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P2 e P'a{h2ll2) 



M Figure 8 p' = g\g 2 ... g-mQ with a finite {H, i)-decomposition. 


B For a limit ordinal a: 

p'M = n (11) 

0<a 

Let us comment the case a +1. When 7L = 0, we have = P^(h'w'). Hence we 

recover the previous situation of weak SPEs. Using different sets H C H(^h'v') and 
allow to have sets E^(hu) bigger than Ea(hu), and thus more plays removed from P'^{hv) 
than in Pa{hv). This situation will be illustrated in Example 28 hereafter. 

The sequence {P'i^{hv))a is nonincreasing by definition, and reaches a fixpoint in the 
following sense (the proof is the same as for Proposition [T7| . 


► Proposition 26. There exists an ordinal /3* such that P'p^{hv) = P'p^j^-^{hv) for all histories 
hv G Hist(uo). 


Our Folk Theorem for SPEs is the next one. The second statement requires to work with 
upper-semicontinuous cost functions A^, i G H. 


► Theorem 27. Let {G,vo) be a quantitative game. 

H If there exists an SPE in (G,vo) with outcome p, then P'p^{hv) 0 for all hv G Hist(uo), 
and p G P^^(uo). 
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H Suppose that all cost functions Xi are upper-semicontinuous. If (hv) ^ 0 for all hv G 
Hist(z;o)7 then there exists an SPE in with outcome p, for all p G P^^(t'o). 

► Example 28. Before proving this theorem, we illustrate it with the example of Figure 
by showing that this game has no SPE (as stated in [IS]). Let us compute the sets P(j(/iri) 
and let us show that 'P''^{hv2) = 0 - By the first statement of Theorem 27 we will get that 
there is no SPE. 

We have to do computations that are more complex than the ones of Example due to 
the usage of sets with H C ^{h’v'), instead of P aih' v’). Clearly, by definition 

Pq(/iv) = Po(/ii^) for all hv, and we have P'^{hv2) = {v^} and P'^{hv3) = {1^3 } for all a as 
in Example [TOj 


Let us first illustrate Definition 24 with the computation of T)q’'^{vo) with H = {ripUiUo, 
(r)or’i)^uo}. The play = {voVi)'^voV2 has a {H, 2 )-decomposition gig2Q'i with 51 = 52 = vqVi 
and g' = V2 (two deviation steps). Indeed p^ has a uot'it’o-deviation step from vqViv'^ G 
P(j,(wo) and a (uoui)^t!o-deviation step from {voviyv^ G P(,(wo^'iw), Last(gi),Last(52) G V2, 
and g[ G P(,((r)or’i)^uo). This play p[ belongs to P>q’^{vo) because its (iL, 2 )-decomposition 
uses the two possible steps of H. On the contrary p2 = has a {H, 2 )-decomposition 

51^32 with £>2 = vqViv^, and does not belong to D^’^(tio) because p'l^p'2 = 5152- One can 
check that = {vqv'^,voViVqv‘^} \J {{vqViY g' \ g' G i;o)}. 

We can now detail the computation of P'^(hvQ) and P'^{hv\). 

B a = 0 . For history hvo, we have Eq(/iuo) D Eo(/ir'o) = {(r;oi'i)“} 0 {voVi)~^voV2 because 
'DQ’\h'v') = PQ{h'v') = Po{h'v') when 77 = 0 . In fact, one checks that Eq(/iuo) = 
Eo(/iuo), and thus 


P[{hvo) = Pi{hvo) = {'yo^^2 } U (uoUi)+U3 . 


For instance, if we try to remove ^^0^2 with cost ( 1 , 2 ) from PQ(hr;o), we have to use some 
H such that 'DQ^{hvoVi) C vi{voVi)*v^ (with cost ( 0 , 1 )). Such a H does not exist. For 
history hvi, we also have Eg(hz;i) = Eo(/ii)i) and 

P[(hvi) = Pi{hvi) = Vi{voVi)*V3. 


H a = 1 . Again we have 

P2(^W) = P2ihvo) = {voVi)^v‘^, P2{hvi) = P2ihvi) = ViivoViYv'^. 
H a = 2 . A difference appears at this step: P3(/iz;i) = t!i(r!oz^i)*z^3 whereas 

P'sihvi) = 0 . 


Indeed Pi'2{hvi) = P'2{hvi). Consider for instance p = ViV^ G P2{hvi), and H = 
hviVo{viVo)~^ C H( 7 r'iWo)- Then (hviVo) has a unique play p' = (voVi)‘^, and 
A2(M > ^2(hvip'). 


Proof of Theorem 1271 We begin by the first statement. Let a be an SPE. As in the proof 
of Lemma 20 let us show by induction on a that (Sfhjy G P'^(hv) for all hv G Hist('(;o). 

For a = 0 , we have (liffijv G P'^(hv) by definition of PQ(hv). 

Let a + 1 be a successor ordinal. By induction hypothesis, we have that (rif/ijv G P'^(hv) 
for all hv. Suppose that (tffft,)„ ^ P'^^i{hv), i.e. {aih)v G E'^{hv). This means that there 
is a history h' = hg G Hist^ for some i with hv < h' < hp, there exists a vertex v' with 
Ev' -ft hp, and there exists H C H(h'w'), such that Vp' G D^’^{h'v'), 

Xi{h-{aih)v) > Xi{h'p). 


( 12 ) 
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Let us consider player i strategy a[ in the subgame {G\h,v) such that a' coincide with 
except that 

(j'iih'i) = v'l, for all hh\v'i G iL U {h'v'}. (13) 


Let gp* = (ct',W e get that p* has a maximal (iL,j)-decomposition such that 
each pn-i G of Definition 24 is equal to (this play belongs to 


P'^{hn-iUn-i) by induction hypothesis). Each deviation step of p* in the sense of Definition 24 
corresponds to a deviation step of p* in the sense of Definition Moreover the {H, i)- 
decomposition of p* is finite (resp. infinite) iff a[ is hnitely (resp. infinitely) deviating from 
a^h- Thus this play p* belongs to D^’*(/i'u'), and by ( [T^ we get Ai(/i-((Tfft)„) > Xi{hgp*). 
Hence tr' is a prohtable deviation for player i in (5f;i,u), a contradiction with a being an 
SPE. 

Let q ; be a limit ordinal. By induction hypothesis {iTih)v G P^(/i?;),V/3 < a. Therefore 
^lh)v G P^ihv) = P'pihv). _ 

Let p G P|^^(uo). By Propos- 


Let us now turn to the second statement of Theorem 


27 


ition [141 if is enough to construct a very weak SPE a with outcome p. The proof is very 
similar to the one of Lemma [21] where the construction of a is done step by step thanks to a 
labeling 7 of the histories. We briefly recall this proof and insist on the differences. 

Initially, no history is labeled. We start with the play p G P^^(t!o), d is partially defined 
such that (d’)„o = p, and 7 (^ 1 ) = p for all non-empty prefixes h of p. 

At the following steps, let h'v' be a history that is not yet labeled, but such that h' 
has already been labeled. Suppose that Last(/i') G Vi. By induction, ')(h') = {a\h)v such 
that hv < h', and {a\h)v G P'p_^{hv). Since P|^^(/iu) = P'i^_^_^_.^{hv) by Proposition 26 we 
have {d'\h)v ^ (^'^)' Therefore, with iL = 0 and D^’*(/i'w') = P'p^{h'v'), we know that 

there exists a play p' G P'^^{h'v') such that Xi{h- {(Tih)v) < Xi{h'p'). Hence, we continue 
to construct tr such that {d\h')vi = and all non-empty prefixes g of p' are labeled by 
'){h'g) = p'. And so on. 

The constructed d is a very weak SPE as in the proof of Lemma ◄ 


The next proposition states that for cost functions that are upper-semicontinuous, sets 
P^, (hu) and Pq., (hv) are all equal. This no longer the case as soon as one cost function is 
not upper-semicontinuous as shown by Examples and 


► Proposition 29. Let (G,vo) be a quantitative game such that all its cost functions are 
upper-semicontinuous. Then for all hv G Hist(uo), P'i^^(hv) = Pa,(hv). 


Proof. Let us prove by induction on a that P'^(hv) C Pa(hv) for all hv. These two sets 
are equal for a = 0. Let a -I- 1 be a successor ordinal and suppose that P'^{hv) C P„(/iu). 
We have 'E'^(hv) D Paihv), and thus P'^_^_i(hv) C Pa+i(hv), because D^’*(h'u') = P'^(h'v') 
when iJ = 0. Eor a being a limit ordinal, we easily have P(j(dv) C Pa(hv) by induction 
hypothesis. 

Suppose now that P'p_^{hv) C Pa^{hv) for some hv. Let p G Pa,{hv) \ P'i^_^{hv), and 
consider the initialized game {G',v'q) = {G\h,v). Notice that the sets Pa{h'v') and Ea{h'v') 


of this game (G', v'q) are exactly the sets Pa(hh'v') and Paihh'v') of (G\h, v)- By Theorem 18 


there exists a weak SPE d in (G'^ v'q) with outcome p. Since the cost functions are upper- 
semicontinuous, a is also an SPE by Proposition 14 Therefore, p G P'p^(hv) by Theorem 
which is a contradiction. 


27 


^^History h'v' leads to one additional deviation step of p* in the sense of Definition(see |13l). 
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Quantitative Reachability Games 


In this section, we focus on quantitative reachability games. Recall that in this case, the 
cost of a play for player i is the number of edges to reach his target set of vertices Ti (see 
Definition!^. Recall also that for quantitative reachability games, SPEs, weak SPEs, and 
very weak SPEs, are equivalent notions (see Corollary 15). 

It is known that there always exists an SPE in quantitative reachability games mm- 

► Theorem 30. Each quantitative reachability game {G,vo) has an SPE. 


As SPEs and weak SPEs coincide in quantitative reachability games, we get the next 
result by Theorem [Tsl 

► Corollary 31. Let {G,vo) be a quantitative reachability game. The sets 
non-empty, for all hv G Hist(uo), and Pq^(uo) is the set of outcomes of SPEs in (G,vo). 

The proof provided for Theorem |30| is non constructive since it relies on topological 
arguments. Our main result is that one can algorithmically construct an SPE in (^,Uo) that 
is moreover finite-memory, thanks to the sets Pq^(/iu). 


► Theorem 32. Each quantitative reachability initialized game (G,vo) has a finite-memory 
SPE. Moreover there is an algorithm to construct such an SPE. 


We can also decide whether there exists a (finite-memory) SPE such that the cost of its 
outcome is component-wise bounded by a given constant vector. 

► Corollary 33. Let {G,vo) be a quantitative reachability initialized game, and let c G 
be a given \T\\-uple of integers. Then one can decide whether there exists a (finite-memory) 
SPE a such that K{{d')vo) < Ci for all z S 11. 

The main ingredients of the proof of Theorem [32| are the next ones; they will be detailed 
in the sequel of this section. We will give afterwards the proof of Corollary [33} 


H Given a, the infinite number of sets Pa(/iu) can be replaced by the finite number of sets 
P^(u) where I is the set of players that did not reach their target set along history h. 

H The fixpoint of Proposition 1 1 7| is reached with some natural number a* G N. 

B Each Pq(u) is a non-empty w-regular set, thus containing a “lasso play” of the form h-g‘^. 
H The lasso plays of each P^(w) allow to construct a finite-memory SPE. 

The next lemma highlights a simple useful property of the cost functions Xi used in 
quantitative reachability games. The proof is immediate. 

► Lemma 34. Let z G 11 and p G Paihv). If player i did not reach his target set along 
history h, then \i{hp) = \i{p) + \hv\. 

The next proposition is a key result that will be used several times later on. It states 
that it is impossible to have plays in Pa(hv) with arbitrarily large costs for player z, without 
having a play in Pa{hv) with an infinite cost for player z. 

► Proposition 35. Consider Pa{hv) and z G 11. If for all p G Pa{hv), we have Xi{p) < -l-oo, 
then there exists c such that for all p G Pa(hv), we have Xi{p) < c. The constant c only 
depends on Pa{hv) and player i. 
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Proof. Suppose that for all n G N, there exists pn G Pa{hv) such that Xi{pn) > n. By 
Konig’s lemma, there exists p = Pn^, for some subsequence {pnk)k of {pn)n- By 

definition of \i in quantitative reachability games, we get \i{p) = +oo. Let us prove by 
induction on a that p G Pa{hv); this will establish Proposition |35| 

Let a = 0. As each G Po(hu), then p^ is a play in {G\h,v) by definition of Po{hv) 
(see Q in Definition 16). Therefore p is also a play in {Gihjv), and p G Po{hv). 

Let a+ 1 be a successor ordinal. As for all n, pn G Pa+i(/iw) C Pa{hv), then p G Pa{hv) 
by induction hypothesis. Let us prove that p G Pa+i{hv). Suppose on the contrary that 
p G Ea{hv) (see Definition 16 and Figure]^. Then there exists a history h' G Histj with 
j G n and hv <h' < hp, there exists a vertex u' with h'v' -ft hp, such that Vp' G Pa{h'v'): 


Xj{hp) > Xj{h'p'). 


(14) 


It follows that player j did not reach his target set along h'. Hence by Lemma 34 we have 
Xj{hp) = Xj{p) + \hv\ and Xj{h'p') = Xj{p') + \h'v'\. By (14), Xj{p') is bounded. Hence by 


induction hypothesis with Pq,(/i'u') and j G H, there exists a constant c such that Xj{p') < c, 
Vp' G PUh'v'). 

Suppose first that Xj{p) < +oo. Then, since p = limk^oo Pn^.^ it follows that for a large 
enough , the plays p and pn^ share a long common prefix on which player j reaches its 
target set, i.e. Xj{p) = Xj{pnJ- It follows that with the same history h'v' as above, by (14) 


and Lemma 34 we have Xj{hpnk) > Xj{h'p'), Vp' G Pa(h'u'), showing that pn^ G Ea(/iu), a 


contradiction. 

Suppose next that Xj{p) = +oo. Then, given c' = \h'v'\ — \hv\ + c, we can choose a large 
enough Uk such that the plays p and Pn^. share a common prefix of length at least d. 
Moreover, as Aj(p) = +oo, player j does not reach its target set along this prefix, i.e. 
Xj{pnf.) > d. Therefore, using the same history h'v' as above, by (14) and Lemma 34 
we have Xj{hpn^,) > \hv\ + d = \h'v'\ + c > Xj{h'p'), dp' G Pa{h'v'). This shows that 
p„j, G Eq,(/iz;), again a contradiction. 

Let a be a limit ordinal. As for all n, p„ G Pa{hv) = p{hv) (see (1^ in Defin- 

16), then p G Pp(/iu),V/3 < a, by induction hypothesis. Hence p €Pa{hv) = 


ition 


n/3<aP/3(H- 

We have just shown that if all plays p G Pa{hv) have a cost Xi{p) < +oo, then there exists 
c such that Ai(p) < c for all such p. This constant c depends on Pa{hv) and player i. ◄ 


As a consequence of Proposition 35 we have that sup{Ai(p) | p G PQ(hu)} is equal to 
max{Ai(p) I p G Pa(/iu)}, and that this maximum belongs to N U {+oo}. 


4.1 Sets Pa{v) 

Let {G, Vo) be a quantitative reachability game. Given a history h — ho ■. ■ hn in (G, vo), we 
denote by I{h) the set of players i such that dk, 0 < k < n, we have hk ^ Tj. In other words 
I{h) is the set of players that did not reach their target set along history h. If h is empty, 
then I{h) — H. The next lemma indicates that sets Pa{hv) only depend on v and I{h), 
and thus not on h (we do no longer take care of players that have reached their target set 
along h). 

► Lemma 36. For hiv, h 2 V G Hist(?;o), if I {hi) = /(/12); then Pa{hiv) = Pa{h2v) for all a. 

Proof. The proof is by induction on a. By definition, we have Po(^i'*^) = Po{h 2 v). 

Suppose that a + 1 is a successor ordinal. By induction hypothesis, Pa{hiv) = Pa{h 2 v). 
Let us prove that Paihiv) = Ea(/i 2 u) which will imply that Pq,_|_i(/iiu) = Pa+i{h 2 v). If 
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p G Ea(hiv), it means that there exists a history h{ = hig G Histi with < hip, there 
exists a vertex v' with h[v' h\p, such that Vp' G Pa(/iiu')i have \i{hip) > Xi{h'ip'), 

i.e. 


Ai(p) > Xi{gp'). 


(15) 


In particular, i G I{h[). Let us consider the history ft ,2 = ^ 25 - By hypothesis, I{hi) = /(/ 12 ), 
and therefore I{h[) = /(/ 12 ) and i G Thus by induction hypothesis Pa{h'iv') = 

Pa{h2v'). It follows that for Vp' G Pa{h2v'), we have Ai(/i 2 p) > Xi{h 2 gp') = Xi(h'2p') by 
(15), and then p G Ea{h 2 v). Symmetrically, if p e Eq(/i 2 'c), then p G Ea{hiv). We can 
conclude that Pq+i(/iiu) = Pq,+i(/i 2 '*^)- 

Suppose that a is a limit ordinal. As Pa{hiv) = np<a and Pp^hiv) = Pp(/i 2 u) 

by induction hypothesis, it follows that Pa{hiv) = Pa{h 2 v). ◄ 


Thanks to this lemma, we can introduce the next definitions. 


► Definition 37. Let {G,vo) be a quantitative reachability initialized game. Let J C n be 
such that I = I(h) for some h G Hist(no). We denote by 
H Pq(u) the set Pa{hv), and by 
H Ea{v) the set Ea{hv). 


In particular, Pa(no) = Pa(uo) and £^( 1 ) 0 ) = Ea(uo). 

Given a, the infinite number of sets Pa{hv) can thus be replaced by the finite number of 
sets Pq(u). Moreover, Proposition 35 can be rephrased as follows. 


► Corollary 38. Consider P^(n) and i G I. If for all p G Pq(u), we have Xi{p) < + 00 , then 
there exists c such that for all p G P^(u), we have Xi{p) < c. The constant c only depends 
on a, I, V, and i. 


Proof. Let h be such that I = I{h). Consider Pa{hv) = P^(f), and i G I. By Proposition!^ 
if for all p G Pa(hv), Xi(p) < + 00 , then there exists c (depending on Pa{hv) and i) such 
that for all p G Pa{hv), Xi{p) < c. By Lemma 36 c depends on a, /, u, and i. ◄ 


As a consequence of Corollary |38[ we give the next definition that indicates the maximum 
costs for plays in P^(u). 

► Definition 39. Given P^^{v), we define A(P^(n)) such that 


A,(Pi(^)) 


— 1 ii i ^ I, 

max{Aj(p) I p G Pa(?^)} if f G /. 


In this definition, —1 indicates that player i has already visited his target set and the 
max belongs to N U {+ 00 }. 


4.2 Fixpoint with a* G N 

In this section, we aim at proving that the fixpoint (when computing the sets P^(u), see 
Proposition [T7|) is reached in a finite number of steps, that is a* G N. 

We first need to introduce some notions about the sets Pq(u). Let p = popi... G Pa(u). 
We use a map y that decorates each pn by some set J C n. The aim of the decoration x{Pn) 
is to indicate at vertex p„, which players of / did not reach their target set along p<n- More 
precisely, x(Pn) = I H I(p^n)- In particular x(Po) = / H 11 = 7. 
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Let Pa{v) and {v,v') G E. We now adapt Definition 


39 to mention the maximum costs 


for plays in Pq(u) starting with edge {v^v'). We define A(P^(u),u') as follows: 

A fP^ (v) v') = I a i ^ I, 

* “ ’ \ max{Aj(p) I p G Pa(u) and popi = vv'} if i G /. 


(16) 


In this definition, the max is equal to -1 when the set {Ai(p) | p G Pq(u) and poPi = vv'} is 
emptyp^ 

The sequence (A(P^(u),is nonincreasing for the usual component-wise ordering 
over (NU { — 1, -foo})'^ since P„(u) is nonincreasing for the inclusion by definition. Therefore 
it reaches a fixpoint that we want to relate to the fixpoint P^ (w) of Proposition 
done in the following lemma. 


17 


This is 


► Lemma 40. ■ //P^(u) = t/ien/or all (u, u') G i?, A(P^(u), u') = A(P^_,_^(u), u')- 

H //P^(u) P^_|_]^(u), then there exist J C n and {u,u') G E such that E{P'^{u)^u') ^ 

A(P^+i(u),m')- 

Proof. The first statement is immediate from definition of A. Let us prove the second 
statement. Consider p = poPi • ■ ■ G E^(u). Then there exist z G 11, n G N and v' ^ Pn+i with 
Pn G Vi, x(Pn) = J, XiPn+l) = J' , SUCh that Vp' G Pa{v') WB have Ai(p) > Xi{po ■ ■ ■ PnP') 
or equivalently (by Lemma 341 

Ai(p) - (n-l-1) > Ai(p')- (17) 

(see the definition of Ea{hv) with I{h) = / in Definition 16 and Figure]^. Notice that i G J'. 
Let us prove that A(P i{u),u') ^ A(P a+l(u),M') with U = Pn and u' = Pn+l- As p G P^(u), 



■ Figure 9 p G E(,,(r), with pn = u and pn+i = u'. 

then p>„ G P^(it) by Lemma 22 As Ai(p>„) = Xi{p) — n, this implies that 
> Xi{p) - n. 


(18) 


Let Q G Pa{u) be such that p starts with edge {u,u') and has maximal cost Ai{P'^{u),u'). 
One gets 

Xi{g) = Ai{Pi{u),u') > Xi{p) - n> Xi{pnp') 


^^Notice that as Pa(r) is non-empty, there exists some {v,v') £ E such that this set is non-empty. 
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by (17) and (18). By considering the set P;^ (v') in Figure]^ it follows that g G E;^(m) for all 
such plays g. Hence C Pq(u) and Ai(Pj^_|_;^(u), u') < Ai(P;;^(u), u'). This completes 

the proof. ◄ 


We are now able to prove that the ordinal a* of Proposition 17 is an integer. 


► Corollary 41. There exists an integer such that for all V G V and 

/ c n. 


Proof. Notice that there is a finite number of sequences (A(P^(ti), u'))q, since they depend 
on / C n and (v,v') € E. As the component-wise ordering over (N U {—1,-|-oo})'^ is a 
well-quasi-ordering and all these sequences are nonincreasing, there exists an integer (and not 
only an ordinal) a(, such that A(P^, (v),v') = A(P^, _f_^(v), v') for all / C H and (v,v') € E. 
By Lemma [40{ we get that a, < «(., showing that a, G N. ◄ 


4.3 The sets ai'e tu-regular 

In this section, we prove that each set Pq(u) is cu-regular. Instead of providing the construction 
of a Biichi automaton (which would lead to many technical details), we prefer to show that 
each set P„(u) is MSO-definable. It is well-known that a set of w-words is w-regular iff it 
is MSO-definable, by Biichi theorem m- Moreover from the Biichi automaton, one can 
construct an equivalent MSO-sentence, and conversely. One can also decide whether an 
MSO-sentence is satisfiable im. We recall that MSO-logic uses: 

H variables x,y,... {X, Y,... resp.) to describe a position (a set of positions resp.) in an 
w-word p, and relations X(x) to mention that x belongs to X, 

H relations Qu(x), u G V, to mention that such vertex u is at position x of p, 

H relations x < y and Succ{x, y) to mention that position y is after position x, and position 
y is successor of position x respectively, 
m, connectives V, A, ^ and quantifiers Bx, Vx, 3X, \/X 

Recall that constants 0,1,... are definable. We will use notation x -I- 1 (and more generally 
X -I- c, with c a constant) instead of Succ{x, y). 

► Proposition 42. Each Pq(x) is an co-regular set. 

We begin with a lemma that states that if P^(u) is w-regular, then the maximum of its 
costs is computable. 

► Lemma 43. //P^(u) is MSO-definable, then A(P^{v)) is computable. 

Proof. Before proving this lemma, we need to establish two properties. The Hrst one states 
that one can decide whether P^(u) has a play p with a given cost for a given player. The 
second one states that when Ai(P^(u)) is finite, then this number is bounded by the number 
of states of a Biichi automaton accepting Pq(u). 

(i) Let c G N U {-boo} and i G I. Let (j) be an MSO-sentence defining Pa(u). Let us 
show that one can decide whether P^(u) has a play p with cost Xi{p) = c. There exists 
an MSO-sentence ip expressing that Xi{p) = c. Indeed, if c = -boo, then p is the sentence 
\/x-^{\/ueTiQu{x)), and ifc < -boo, it is the sentence (Vx < c-^{\/ueTiQu{x)))A(yueTiQuic)). 
Therefore one can decide whether the MSO-sentence A p is satisfiable by some play p. 

(a) Let i G I and suppose that Ai{Pf{v)) < -boo. Let H be a Biichi automaton accepting 
P^(w). We now show that Ai(P^(u)) < n where n is the number of states of B. Assume 
the contrary and consider an accepting run r = rori ... of H on a play p = poPi •. • G Pa(u) 
with Xi{p) = Ai(P^(u)) > n. The prefix r<„ of r has a cycle rk .. .ri with 0 < k < I < n 
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and rfc = r;. This cycle can be repeated once, while keeping an accepting run labeled by 
p' = po • ■ ■ (/Ofc • ■ ■pi-i)‘^p>i. As A*(p) > n, it follows that \i{p') = Ai(P^(n)) + {I - k). 
Therefore we get a contradiction with Xi{p) = Ai(P^(r;)). 

Let us prove the lemma. By definition Ai(P^(w)) equals —1 ii i ^ I, and is thus 
computable in this case. Let i € I. By (i), one can decide whether Ai(P^(u)) = +oo. 
In case of a positive answer, Ai(P^(u)) is thus computable. If the answer is negative, as 
Ai(P^(u)) < n by (ii), we can similarly test whether Ai(P^(u)) = c by considering decreasing 
constants c from n — 1 to 0. This prove that A(P^(u)) is computable. ◄ 


Proof of Proposition |42[ Let us prove that each set Pa('y) is MSO-definable by induction 


on a. 


For a = 0, recall that Pq{v) is the set of plays starting with v. The required sentence is 
thus Qy{0) A Vx ■ \/(u,u')eE{Qu{x) A Qu'{x + 1)). 

Let a e N be a fixed integer. By induction hypothesis, each set P^(u) is MSO-definable, 
and by Lemma 43 A(P^(u)) is computable. These sets and constants can be considered 


as fixed. Let us prove that E^(u) is MSO-definable. It will follow that Pq+i(u) is also 
MSO-definable. Thanks to A(P^(u)), the definition of p G E^(u) can be rephrased as follows; 
there exist n G N, i G /, and u,u',v' G V with u' ^ u', {u,v') G E, such that pn = u G Vi, 
Pn+i = u' , x(p„+i) = J', and 

A(p) >A,(Pf(u')) + (n+l) (19) 


(see Figure]^. Notice that (19) implies that i £ J' and Ai(P;( (u')) < -foo. Moreover 
Ai(P;( {v')) is a fixed integer. 

Let us provide an MSO-sentence ip defining E^(u). The next sentence expresses 
that J' C I is the subset of players of / that did not visit their target set along p<„: 


— (Vx * (x ^ Tij ^ ~'(\/j^jf (^^.(x))) a (Aj^/^j/dx ^ TL * \/. 


The next sentence expresses that if player i visits its target set along p, it is after 

Ai(P;^ (u')) -I- n -I- 1 edges from po: 

^J',n,v>,i = Vx • (yreT.Qrix) (^X > A,(P^'(u')) -f U-f 1^ ^ 

Notice that in the previous formula, Ai(P;^ (u')) is a constant since P;( (v') is a fixed set. 
The required formula tp is then the following one: 

3n- V V V {Qu{n) A Qu'{n+ 1) A (j>jpn A (pj',n,v',i) ■ 

u^u'^v'^V t/'C/ i£j' 

(u,v')gE AiiFp'{ v'))<+oo 




By Proposition 42 the next corollary states that one can effectively extract a lasso play 
from P^(u) that has a maximal cost for a given player i. 

► Corollary 44. For all i G I, each set P^^(x) has a computable lasso play h-g'^ with 
Xi{h-g‘^) = Ai(P^^(u)). This play depends on i, I, and v. 


Proof. Part (i) oi the proof of Lemma 43 indicates that the set of plays p G P„ (u) with 


maximal cost Ai(p) = Ai(P^ (u)) is w-regular. Therefore, from a Biichi automaton accepting 
this set, we can extract an accepted lasso play of the form h-g‘^ with the required cost. Such 
a play depends on i, I, and v (a* is fixed). ◄ 
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4.4 Construction of a Finite-Memory SPE 


Thanks to the results of Sections 4.1|4.3| we have all the ingredients to prove that each 
quantitative reachability game has a computable finite-memory SPE. 


Proof of Theorem 1321 Let (5,no) be a quantitative reachability game. Let us summarize 
the results obtained previously. By Corollary 


31 


each set (v) is non-empty with v G V 
and I = I{h) for some history hv G Hist(uo), and Pjj (ug) contains all the outcomes of SPEs 


in (^,Uo). By Corollary 41 and Proposition we know that a, G N and each P^^(u) is an 
w-regular set that can be constructed. Finally by Corollary]^ for all i G I, one can construct 
a lasso play G Pa,('y) with maximal cost = Ai(P^Ju)). 

We now show how to construct a finite-memory SPE a from the finite set of lasso plays 
hij^v‘{gi,i,v)^■ The procedure is similar to the one developed in the proof of Theorem 18 and 
more particularly of Lemma We indicate how to adapt the proof of this lemma. Again 
the construction of a is done step by step, thanks to a labeling 7 of the non-empty histories. 

Initially, none of the histories is labeled. We start with history ug and with any play 
€ Pjj^(ug), i G n. The strategy profile a is partially defined such that 
(d)„o = hi^-n_,vo-{gi,n,voY^ and the non-empty prefixes h of hi^YL,vo-{9i,n,voY are all labeled 
with 7 (/i) = (z,n,'(;o). 

At the following steps, we consider a history h'v' that is not yet labeled, but such that h' 
has already been labeled. By induction, 7 (/i') = {j,I,v) and there exists hv < h' such that 


{h\h)v = Suppose that Last(h') G Vi and I{h') = J' , the proof of Lemma 21 

requires to choos^^a play p' G pT^(z;') such that \i{h' p') > Xi{h{d\h)v) (see (j^). We simply 
choose p' = hijpy! ■{gijpyi)‘^ that has maximal cost Xi{hijiy {gijiy)‘^) — Ai(P;(^ (w')). 
Then we continue the construction of a such that {a\h')v' = hi^ji^y> ■(gi^j>y)‘^, and for all 
non-empty prefixes g of hi^j/ y{gi^jpy')‘^ , we define ^{h' g) = [i, J',v'). 

By the proof of Lemma |2H the strategy profile a is an SPE. It is finite-memory since 
for all h G Histi, ai{h) only depends on ^{h) = and hjj^y-{gjj^y)‘^. There is a finite 

number of lasso plays hjj^y-{gjj^y)‘^, and j{h) (as well as 1(h)) can be computed inductively 
as follows. Initially, I(vo) = 11, and 7 ( 1 ^ 0 ) = (t,n,ug) for some chosen i G 11. Let h' G Histi 
and suppose that I(h') = J' and ')(h') = (j,I,v). Then I(h'v') = J' \ {i | u' G Ti}. If h'v' 
respects hjjy{gjj^y)‘^, i.e. ai(h') = v', then 7 (h'v') = {j,I,v). Otherwise 7 (h'u') = {i,J',v') 
with (i, J',v') computed as in the previous paragraph. ◄ 


4.5 Constrained Existence 

It remains to prove the decidability of the constrained existence of SPE for quantitative 
reachability games, as announced in Corollary This result is easily proved on the basis of 
some previous properties. 

Proof of Corollary |33[ Let {G,vo) be a game and let c G be a constant vector. In the 
proof of Lemma we have seen that there exists an MSO-sentence expressing that play p 
has a fixed cost Xi{p) = ct. Similarly, one can express that Xi(p) < Ci by the next sentence (^7 
3a; < Ci • (Vy^TiQu(x)). By Proposition 42 we know that the set Pjj^(ug) of outcomes of 
SPEs in (G, vq) is an w-regular set, and that one can construct an MSO-sentence (p defining it. 
Therefore the set of outcomes of SPEs with a cost component-wise bounded by c is definable 
by Xiyipi A (j), and is then w-regular. Moreover, one can decide whether this set is non-empty. 


This proof states that such a play always exists. 
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In case of positive answer, it contains a lasso play h-g‘^. Exactly as done in Section \4^ one 
can construct a finite-memory SPE a such that {d')yg = h-g'^. This concludes the proof. ◄ 


5 


Games with Prefix-independent Regular Cost Functions 


In this section, we present a class of games for which it is decidable whether there exists a 
weak SPEThe hypotheses are general conditions on the cost functions Ai, i S 11: each 
function \i must be prefix-independent (see Definition]^, Ai has to use a finite number 
of values (gathered in set Ci), and the set of plays p with a given cost \i{p) = Ci must be 
w-regular. 

► Theorem 45. Let {G,vo) be an initialized game such that: 

H each cost function A^ is prefix-independent, and with finite range Ci C Q, 

H for all i G H, Ci G Ci, and v G V, the set of plays p in {G,v) with Xi{p) = Ci is an 
Lo-regular set. 

Then one can decide whether {G, vq) has a weak SPE a (resp. such that Xi{{a}yg) < Ci forall i 
for given Ci G Ci, i G H). In case of positive answer, one can construct such a finite-memory 
weak SPE. 


For example, the hypotheses of this theorem are satisfied by the liminf games and the 
limsup games; they are also satisfied by the game of Example [T^ We will see that the proof 
of this decidability result shares similar points with the proof given in the previous section 
for quantitative reachability games. Again, we will use the Folk Theorem for weak SPEs (see 
Theorem 18) to prove this result. The main steps of the proof are the following ones. 

H Given a, the infinite number of sets Paihv) can be replaced by the finite number of sets 
Pa(l^). 


H The fixpoint of Proposition |17| is reached with some natural number a* G N. 

H Each Pa('c) is an w-regular set. Therefore there exists an algorithm to construct the 
sets Pa. (v) for all v gV, and thus to decide whether they are all non-empty. For given 
constants Ci G Ci, t G 11, one can also decide whether Pa. (uq) has a play p with bounded 
cost Xi{p) < Ci for all i. 

H In case of positive answer, some lasso plays of the sets Pa.(n) allow to construct a 
finite-memory weak SPE (resp. with bounded cost). 


To establish Theorem 45 we prove a series of lemmas. The first lemma states that Pa{hv) 
is independent of h. There is thus a finite number of sets Pa(n), v G V, to study. 


► Lemma 46. Pa{hv) = Pa(n) for all hv G Hist(r:o). 


Proof. The proof can be easily done by induction on a. It uses the definition of Ea{hv) and 
the hypothesis of Theorem [45] that each cost function Xi is prefix-independent. ◄ 


As each cost function Xi is supposed to have finite range in Theorem |45[ we can give the 
next definition that indicates the maximum costs for plays in Pa(r') (resp. starting with vv', 
for some given (y,v') G E). Recall that a similar definition was given in case of quantitative 
reachability games (see Definition 39 and @). 


► Definition 47. Given Pa('c) and {v,v') G E, we define for each i G 11: 


14 


Contrarily to quantitative reachability games, we do not know if a weak SPE always exists for games 
in this class. 
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■ Aj(Pa(i;)) = max{Ai(p) I p G Pa(i^)}. 

■ Aj(Pq(i;),u') = max{Ai(p) | p G Pa(f) and poPi = w'}. 


In this definition, the max is equal to —oo if it applies to an empty set. 

The sequence (A(Pc(u),is nonincreasing for the component-wise ordering over 
(Q U {—oo})l'^l 0 Therefore it reaches a fixpoint as for (Pq(u))q,. The following lemma 
relates these sequences. 


► Lemma 48. h If'Pa{v)='Pa+i{v),thenforall{v,v')&E,K{Pa{v),v')=K{'Pa+i{v),v'). 
H IfPa{v) ^Pa+i{v), then there exists {u,u') G E such that A(Pa(u), u') A(PQ_|_i(tt), w'). 


Proof. The proof is similar to the proof of Lemma Without mentioning it, we will 
repeatedly use Lemma 46 and the hypothesis of Theorem that the cost functions are 
prefix-independent. 

The first statement is immediate from definition of A. For the second statement, consider 
p = poPi • ■ ■ G Ea(n). Then by definition of Ea(n), there exist n G N, i G 11, and u,u',v' G V 
with u' ^ v', (u, v') G E, such that pn = u G Vi, Pn+i = u', and 


Vp' G P„(u') : A,(p) > A,(p'). 

(See Figure adapted to the context of Lemma 48). Let us prove that A(Pq,(m),u') ^ 
A(P Q+i(u),w'). As p G Pq(u), then p>„ G Pa('«) by Lemma |22[ which implies that 


Ai(Pa('u),u') > Ai(p>„). Let g G Pc((m) be such that g starts with edge {u,u') and has 
maximal cost Ai(Po,(?x),u'). One gets 


X,{g) = Ai{Pa{u),u’) > Ai(p>„) = Xi{p) > Ai(p'). 


Hence, using the same set PQ(r'') as for p, it follows that g G Eq(m) for all such plays g. 
Therefore Pq,+i(m) C Pq(u) and AiiPa{u),u') < Ai(PQ+i(u), m'). ◄ 

As a consequence, the ordinal a* of Proposition is an integer. The proof is the same 
as for Corollary [4T| 

► Corollary 49. There exists an integer a* such that Pq,. (u) = Po,,+i(n) for all v GV. 

As done for quantitative reachability games, let us now prove that the sets Pa(u) are 
w-regular for all a and v. 


► Lemma 50. Each Pa(n) is an co-regular set. 


Proof. The proof is similar to the proof of Lemma 43 and Proposition(It is even simpler). 
Recall that as soon as Pa('y) is empty, then Pp('c) = 0 for all /3 > a. 


Like in Lemma 43 we first prove that if Pct('y) is MSO-definable, then Ai(Po,(r!)) is 
computable for each i G H. Let (p be an MSO-sentence defining PQ(n). One can decide 
whether PQ(n) is empty. If this is the case, then Ai(P„(n)) = —oo for all i. Suppose that 
Pa(^^) 7^ 0; and let i G n and c G Q. By hypothesis, the set of plays p in {G,v) with 
cost Xi{p) = c is w-regular and thus MSO-definable by a sentence (pc,i- We can thus decide 
whether Pq-(i') has a play p with cost Ai(p) = c, thanks to sentence (p A Pc,i- Therefore, by 
considering decreasing constants c G Ct, we can decide whether Ai(PQ(u)) = c. This shows 
that Ai(Pa(n)) is computable. 


^®More precisely each component in (Q U {—is restricted to Ci U {—oo}. 
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Let us now prove that each set Pq(z;) is MSO-definable by induction on a. For 0 = 0, 


we use the same defining MSO-sentence as in the proof of Proposition 42 

AVx * ^a Qu' ^))' 


Let a G N be a fixed integer. By induction hypothesis, each set Pa(w) is MSO-definable, and 
Ai(P^(r;)), * G n, is computable by the first part of the proof. These sets and constants can 
be considered as fixed. The only case to consider is Pa(^^) ^ 0 (recall that this property is 
decidable). To show that PQ+i(r’) is also MSO-definable, it is enough to prove that Eq(ii) 
is MSO-definable. Recall that p G Eq,(z)) iff there exist n G N, z G 11, and u,u',v' G V with 
u' ^ v', (u,v') G B, such that pn = u £ Vi, Pn+i = u', and Vp' G Pa(w'): \i{p) > A,;(p'). 
The last condition can be replaced by \i{p) > Ai(PQ(z;')) ^ — oo|^ Let us provide an 
MSO-sentence ip defining Ea(z;): 


3n- y 

iGU,uGVi 

u 

{u,v')gE 


\J ^ Qu'i,"^ -|- 1) A ^c,i') • 

cGCi 




We get the next corollary. The proof is the same as for Corollary 

► Corollary 51. If Paiv) 0, then one can compute a lasso play h-g‘^ in Pa('c) with 
Xi{h-g‘^) = Ai(P„(i;)). This play depends on i and v. 


We are now able to prove the main result of this section. 


Proof of Theorem 1451 Let (Q,vo) be a game satisfying the hypotheses of Theorem 45 
Let us summarize the results of the previous lemmas. We know that a* G N and that 
one can construct the sets Pa,(z)), v £ V. As these sets are w-regular, one can decide 
whether they are all non-empty. In case of positive answer, there exists a weak SPE in 

If in addition some constants Ci G Ci are given, then the set 
Xi(p) < Ci,\/i £ n} is also w-regular. Hence one can also decide 


{G,vo) by Theorem 18 
Pa. (I’d) n {p in {g,vo) 


whether this set is non-empty and thus whether there exists a weak SPE in (G,vo) with cost 
component-wise bounded by c. This establishes the first part of Theorem 

Suppose that such a weak SPE exists, then let us show that we can construct a weak 
SPE that is finite-memory with the same construction as in the proof of Theorem By 
Corollary 51 for all z G H, n G V, one can construct a lasso play hi^vfgi^v)'^ £ Pc, (w) with 
maximal cost Xi{hi^y-{gi^y)‘^) = A.i(Pa, (z;)). The construction of a finite-memory SPE d 
from the finite set of lasso plays hi^yfgi^y)'^ is conducted as in the proof of Lemma 21 It is 
done step by step thanks to a labeling 7 of the non-empty histories. 

Initially, none of the histories is labeled. We start with history vq and with any play 
hi,vo-{gi,voT G Pa.('Co), i G nj^ The strategy profile a is partially defined such that 
{d')vo = hi^yg ■ {gi^yy)^^, and the non-empty prefixes h of hi^yg ■ (gi^yg)‘^ are all labeled with 
7 (/z) = {i,vo). 

At the following steps, we consider a history Bv' that is not yet labeled, but such that 
h' has already been labeled. By induction, 7 (/z') = {j,v) and there exists hv < h' such that 


^®Set Pq(zz^) must be non-empty. 

^^When some constants a £ Ci are additionally given, play /zi.uo' must be replaced by any 

lasso play in Pq.(^o) H {p in (G,vo) | Xi(p) < Ci,Vi £ H}. 
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{^\h)v = hj^v-{gj,vY■ Suppose that Last(/i') G Vi, the proof of Lemma 21 requires to choose 
a play p' G Pq,(u') such that Xi{h'p') = Xi{p') > Xi{h-{d-ih)v)- We choose p' = hi^yi ■{giy)‘^ 
with maximal cost Xi{p') = Ai(PQ,^ (u')). Then we continue the construction of a such 
that {a\h')v' = hiy ■ , and for all non-empty prefixes g of hi^^'■ {giy)'^, we define 

l{h'g) = (i,v'). 

We know by Lemma |21| that a is an SPE. It is finite-memory because it only depends 
on the finite number of lasso plays hj^y- {gj^y)‘^, and the labeling 7 that can be computed 
inductively as follows. Initially, 'y(vo) = (ijVo) for some chosen i G 11. Let h' G Histi and 
suppose that 7 (h') = {j,v). If h’v' respects hj^y-{gj^y)^, then ^{h'v') = {j,v), otherwise 
'){h'v') = {i,v') (as explained in the previous paragraph). This establishes the second part 
of Theorem HU ◄ 


6 Conclusion and Future Work 


In this article, we have studied the existence of (weak) SPEs in quantitative games. We 
have proposed a Folk Theorem for weak SPEs, and a weaker version for SPEs. To illustrate 
the potential of this theorem, we have given two applications. The first one is concerned 
with quantitative reachability games for which we have provided an algorithm to compute 
a finite-memory SPE, and a second algorithm for deciding the constrained existence of a 
(finite-memory) SPE. The second application is concerned with another large class of games 
for which we have proved that the (constrained) existence of a (finite-memory) weak SPE is 
decidable. 

Future possible directions of research are the following ones. We would like to study the 
complexities of the problems studied for the two classes of games. We also want to investigate 
the application of our Folk Theorem to other classes of games. The example of Figure is a 
game with a weak SPE but no SPE (see Example 12). Recall that for this game, the cost 


Aj(p) can be seen as either the mean-payoff, or the liminf, or the limsup, of the weights of p. 
We do not know if games with this kind of payoff functions always have a weak SPE or not. 
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